What is claimed is: 



1 . An image processing device for detemiining three-dimensional 
position and/or orientation of an object, comprising: 

image data capturing means for capturing image data containing an 
image of the object; 

model pattem creating means for creating a model pattem based on 
image data of a reference object with a reference orientation relatively to said 
image capturing means captured by said image capturing means, said reference 
object having a shape substantially identical to that of the object; 

transformation means for performing two-dimensional and geometrical 
transformation of the created model pattem to generate a transformed model 
pattem representing an image of the object with an orientation different from 
the reference orientation; 

pattern matching means for performing a pattem matching of the image 
data of the object captured by said image capturing means with the transformed 
model pattem; 

selecting means for repeatedly performing the generation of a 
transformed model pattem and the pattem matching of the image data of the 
object with the transformed model pattem to thereby select one of the 
transformed model patterns in conformity v^th the image data of the object, 
and obtain information on a position of the image of the object in the image 
data; and 

determining means for determining three-dimensional position and/or 
orientation of the object based on the information on the position of the image 
of the object in the image data and information on the orientation of the 
selected one of the transformed model patterns. 

2. An image processing device for determining three-dimensional 
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position and/or orientation of an object, comprising: 

image data capturing means for capturing image data containing an 
image of the object; 

model creating means for creating a model pattem based on image data 
of a reference object with a reference orientation relative to said image data 
capturing means captured by said image data capturing means, said reference 
object having a shape substantially identical to that of the object; 

transformation means for performing two-dimensional and geometrical 
transformation of the created model pattem to generate a plurality of 
transformed model pattems each representing an image of the object with an 
orientation different from the reference position; 

storage means for storing the plurality of transformed model pattems and 
information on orientations of the respective transformed model pattems; 

pattem matching means for performing pattem matching of the image 
data of the object captured by said image capturing means with the plurality of 
transformed model pattems to thereby select one of the transformed model 
pattems in conformity with the image data of the object, and obtain 
information on a position of the image of the object in the image data; and 

determining means for determining three-dimensional position and/or 
orientation of the object based on information on the position of the image of 
the object in the image data and the information on an orientation of the 
selected one of the transformed model pattems. 

3. An image processing device according to claim 1 or 2, wherein said 
transformation means performs the two-dimensional and geometrical 
transformation of an affine transformation, and said image processing device 
further comprises additional measuring means for obtaining a sign of 
inclination of the object with respect to said image capturing means. 



4. An image processing device according to claim 3, wherein said 
additional measuring means performs dividing of a model pattem into at least 
two partial model pattems which are subject to the afiSne transformation to 
generate transformed partial model pattems, and pattem matching of the image 
data of the object with the transformed partial model pattems to determine 
most conformable sizes, and determines the sign of the inclination based on 
comparison of the sizes of the conformable partial model pattems with each 
other. 

5. An image processing device according to claim 3, wherein said 
additional measuring means performs measurement of distances from a 
displacement sensor separately provided in the vicinity of said image capturing 
means to at least two points on the object using the displacement sensor, and 
determines the sign of the inclination based on comparison of the measured 
distances. 

6. An image processing device according to claim 3, wherein said 
additional measuring means performs additional pattem matching of image 
data of the object captured after said image data capturing means is slightly 
moved or inclined and determines the sign of the inclination based on judgment 
whether an inclination of image of the object becomes larger or smaller than 
the selected one of the transformed model pattems. 

7. An image processing device according to claim 1, wherein the 
image processing device is incorporated into a robot system comprising: 

storage means storing an operating orientation of the robot relative to the 
object or storing an operating orientation and an operating position of the robot 
relative to the object; and 

robot control means for determining an operating orientation of the robot 
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or the operating orientation and an operating position of the robot based on the 
determined three-dimensional position and/or orientation of the object. 

8. An image processing device according to claim 7, wherein said 
image capturing mean? is mounted on the robot. 
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